I am a Research Group Leader at the Max Planck Institute for Informatics. We design methods for digitizing our surroundings. To this end, we use data captured by monocular RGB cameras and other sensors to model, reconstruct and videorealistically render our world. Human faces are at the forefront of our visual communications, and hence their processing is of a special interest to us, with applications in visual telepresence in our mind. Our work lies in the intersection of computer vision and computer graphics, and usually includes a heavy machine and deep learning component through supervised, self-supervised or unsupervised learning. Our end goal is to fully digitize our world to allow all of us to connect and interact with our friends, family, and loved ones, over a distance.

Topics of interest include:  

• 3D scene modeling and reconstruction
• Deep generative models
• Videorealistic neural rendering
• Pose estimation
• Relighting
• Event camreas

If you are interested in Master's thesis, Research Immersion Lab (RIL) or Hiwi positions, feel free to contact us. We are also always looking for motivated PhD students.

HQ3DAvatar: High Quality Implicit 3D Head Avatar K. Teotia, Mallikarjun B R, X. Pan, H. Kim, P. Garrido, M. Elgharib and C. Theobalt In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2024) [Manuscript], [Project Page]

Lite2Relight: 3D-aware Single Image Portrait Relighting P. Rao, G. Fox, A. Meka, M. B R, F. Zhan, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, M. Elgharib and C. Theobalt SIGGRAPH 2024 (conference track) [Manuscript (coming soon)], [Project Page (coming soon)]

Unsupervised Event-based Video Reconstruction G. Fox, X. Pan, A. Tewari, M. Elgharib and C. Theobalt In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024 [Manuscript], [Project Page]

AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars M. Mendiratta, X. Pan*, M. Elgharib*, K. Teotia, Mallikarjun B R, A. Tewari, V. Golyanik, A. Kortylewski and C. Theobalt (* equal contribution) In ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2023) [Manuscript], [Project Page]

A Deeper Analysis of Volumetric Relightable Faces P. Rao, M. B R, G. Fox, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, F. Zhan, A. Tewari, C. Theobalt and M. Elgharib International Journal of Computer Vision 2023 (IJCV - Invited Paper) [Manuscript]

LiveHand: Real-time and Photorealistic Neural Hand Rendering A. Mundra, M. B R, J. Wang, M. Habermann, C. Theobalt and M. Elgharib To appear in the International Conference on Computer Vision 2023 (ICCV) [Manuscript], [Project Page]

EventNeRF: Neural Radiance Fields from a Single Colour Event Camera V. Rudnev, M. Elgharib, C. Theobalt and V. Golyanik In Computer Vision and Pattern Recognition 2023 (CVPR) [Manuscript], [Project Page]

An Implicit Parametric Morphable Dental Model C. Zhang, M. Elgharib, G. Fox, M. Gu, C. Theobalt and W. Wang In ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2022) [Manuscript], [Project Page]

VoRF: Volumetric Relightable Faces P. Rao, M. B R, G. Fox, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, A. Tewari, C. Theobalt and M. Elgharib To appear in the British Machine Vision Conference 2022 (BMVC) - Oral - Best Paper Award Honourable Mention [Manuscript], [Project Page]

gCoRF: Generative Compositional Radiance Fields M. B R, A. Tewari, X. Pan, M. Elgharib and C. Theobalt In the International Conference on 3D Vision 2022 (3DV) - Spotlight Presentation [Manuscript], [Project Page]

Neural Radiance Fields for Outdoor Scene Relighting V. Rudnev, M. Elgharib, W. Smith, L. Liu, V. Golyanik and C. Theobalt European Conference on Computer Vision 2022 (ECCV) [Manuscript], [Project Page]

A Motion Matching-based Framework for Controllable Gesture Synthesis from Speech I. Habibie, M. Elgharib, K. Sarkar, A. Abdullah, S., M. Neff and C. Theobalt SIGGRAPH 2022 (conference track) [Manuscript], [Project Page]

φ-SfT: Shape-from-Template with a Physics-Based Deformation Model N. Kairanda, E. Tretschk, M. Elgharib, C. Theobalt and V. Golyanik. Computer Vision and Pattern Recognition 2022 (CVPR) [Manuscript], [Project Page]

StyleVideoGAN2: A Temporal Generative Model using a Pretrained StyleGAN G. Fox, A. Tewari, M. Elgharib and C. Theobalt British Machine Vision Conference 2021(BMVC) - Oral [Manuscript], [Project Page]

EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream V. Rudnev, V. Golyanik, J. Wang, H-P. Seidel, F. Mueller, M. Elgharib and C. Theobalt International Conference on Computer Vision 2021 (ICCV) [Manuscript], [video], [Project Page], [ICCV Daily]

Learning Speech-driven 3D Conversational Gestures from Video I. Habibie, W. Xu, D. Mehta, L. Liu, H-P. Seidel, G. Pons-Moll, M. Elgharib and C. Theobalt ACM International Conference on Intelligent Virtual Agents 2021 (IVA) - Best Paper Award [Manuscript], [Project Page]

PhotoApp: Photorealistic Appearance Editing of Head Portraits M. B R, A. Tewari, A. Dib, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, L. Chevallier, M. Elgharib and C. Theobalt ACM Transactions on Graphics (Proceedings of SIGGRAPH - 2021) [Manuscript], [Project Page]

High-Fidelity Neural Human Motion Transfer from Monocular Video M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H-P. Seidel, S. Castillo, C. Theobalt and M. Magnor Computer Vision and Pattern Recognition 2021 (CVPR) - Oral [Manuscript], [video], [Project Page]

i3DMM: Deep Implicit 3D Morphable Model of Human Heads T. Yenamandra, A. Tewari, F. Bernard, H-P. Seidel, M. Elgharib, D. Cremers and C. Theobalt Computer Vision and Pattern Recognition 2021 (CVPR) - Oral [Manuscript], [video], [Project Page]

Monocular Reconstruction of Neural Face Reflectance Fields M. B R, A. Tewari, T-H. Oh, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, M. Elgharib and C. Theobalt Computer Vision and Pattern Recognition 2021 (CVPR) [Manuscript], [Project Page]

Learning Complete 3D Morphable Face Models from Images and Videos M. B R, A. Tewari, H-P. Seidel, M. Elgharib and C. Theobalt Computer Vision and Pattern Recognition 2021 (CVPR) [Manuscript], [Project Page]

VideoForensicsHQ: Detecting High-quality Manipulated Face Videos G. Fox, W. Liu, H. Kim, H-P. Seidel, M. Elgharib and C. Theobalt IEEE International Conference on Multimedia and Expo 2021 (ICME) - Oral [Manuscript], [Project Page]

Egocentric Videoconferencing M. Elgharib*, M. Mendiratta*, J. Thies, M. Niessner, H-P. Seidel, A. Tewari, V. Golyanik and C. Theobalt (* equal contribution) ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020) [Manuscript], [Project Page], [Video], [Two Minute Papers!]

PIE: Portrait Image Embedding for Semantic Control A. Tewari, M. Elgharib, M. B R, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020) [Manuscript], [Project Page]

Neural Voice Puppetry: Audio-driven Facial Reenactment J. Thies, M. Elgharib, A. Tewari, C. Theobalt and M. Niessner European Conference on Computer Vision 2020 (ECCV) [Manuscript], [Project page], [Demo], [Source code]

Self-supervised Outdoor Scene Relighting Y. Yu, A. Meka, M. Elgharib, H.P. Seidel, C. Theobalt and W. Smith European Conference on Computer Vision 2020 (ECCV) [Manuscript], [Project page]

XNect: Real-time Multi-person 3D Motion Capture with a Single RGB Camera D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.P. Seidel, H. Rhodin, G. Pons-Moll and C. Theobalt ACM Transactions on Graphics (Proceedings of SIGGRAPH 2020) [Manuscript], [Project page], [Demo@CVPR19], [SelecSLS-Pytorch]

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt Computer Vision and Pattern Recognition 2020 (CVPR) - Oral [Manuscript], [Project page]

Neural Style-Preserving Visual Dubbing H. Kim, M. Elgharib, M. Zollhofer, T. Beeler, H.P. Seidel, C. Richardt and C. Theobalt. ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2019) [Manuscript], [Project page]

FML: Face Model Learning A. Tewari, F. Bernard, P. Garrido, G. Bharaj, M. Elgharib, H.P. Seidel, P. Perez, M. Zollhofer and C. Theobalt. Computer Vision and Pattern Recognition 2019 (CVPR) - Oral [Manuscript], [Project page]

On Learning Associations of Faces and Voices C. Kim, H. V. Shin, T. H. Oh, A. Kaspar, M. Elgharib, and W. Matusik. Asian Conference on Computer Vision 2018 (ACCV) [Manuscript], [Project page]

Learning-based Video Motion Magnification T. H. Oh*, R. Jaroensri*, C. Kim, M. Elgharib, F. Durand, W. T. Freeman, and W. Matusik. *indicates equal contributions European Conference on Computer Vision 2018 (ECCV) - Oral [Manuscript], [Project page]

A Dataset of Flash and Ambient Illumination Pairs from the Crowd Y. Aksoy, C. Kim, P. Kellnhofer, S. Paris, M. Elgharib, M. Pollefeys, and W. Matusik. European Conference on Computer Vision 2018 (ECCV) [Manuscript], [Project page]

Crowd-Guided Ensembles. How Can We Choreograph Crowd Workers for Video Segmentation? A. Kasper, G. Patterson, C. Kim, Y. Aksoy, W. Matusik, and M. Elgharib. ACM Conference on Human Factors in Computing Systems 2018 (CHI) - Oral [Manuscript], [Video teaser], [Project page]

Video reflection removal through spatio-temporal optimization A. Nandoriya*, M. Elgharib*, C. Kim, M. Hefeeda, and W. Matusik. *indicates equal contrbutions. International Conference on Computer Vision 2017 (ICCV) [Manuscript], [Project page]

Sports VR Content Generation from Regular Camera Feeds K. Calagari, M. Elgharib, S. Shirmohammadi, and M. Hefeeda. ACM Multimedia 2017 (ACM MM) - Full Paper [Manuscript], [Project page]

Data Driven 2-D to 3-D Video Conversion for Soccer K. Calagari, M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda. IEEE Transactions on Multimedia 2017 (TMM) [Manuscript], [Project page]

Painting Style Transfer for Head Portraits using Convolutional Neural Networks A. Selim*, M. Elgharib*, and L. Doyle. *joint first author, sorted alphabetically ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016) Selected as the proceedings back-cover! [Manuscript], [Video teaser], [Project page]

Gradient-based 2-D to 3-D conversion for Soccer Videos K. Calagari,M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda. ACM Multimedia 2015 (ACM MM) - Oral Selected as MIT home-page! [Manuscript], [MIT main page], [Project page]

Video Magnification in Presence of Large Motions M. Elgharib, M. Hefeeda, F. Durand and W. T. Freeman. Computer Vision and Pattern Recongition 2015 (CVPR) [Manuscript], [Project page]

Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes G. Castanon, M. Elgharib, P.M. Jodoin and V. Saligrama. IEEE Transactions on Circuits and Systems for Video Technology, 26:12, Dec. 2016 (accepted Aug. 2015) (TCSVT) [Manuscript], [Project page]

Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique M. Elgharib, F. Pitie, and Anil Kokaram. EURASIP Journal on Image and Video Processing 2013:33 (EURASIP) [Manuscript], [Project page]

Motion Estimation for Regions of Reflections through Layer Separation M. Elgharib, F. Pitie, and Anil Kokaram. ACM SIGGRAPH European Conference on Visual Media Production 2011 (CVMP) - Oral [Manuscript], [Project page]

Reflection Detection in Image Sequences M. Elgharib, F. Pitie, and Anil Kokaram. Computer Vision and Pattern Recognition 2011 (CVPR) [Manuscript], [Project page]

Extraction of Non-binary Blotch Mattes M. Elgharib, F. Pitie, and Anil Kokaram. International Conference on Image Processing 2009 (ICIP) [Manuscript], [Project page]

A. Hassanein, M. Elgharib, A. Selim, S. Bae, M. Hefeeda and W. Matusik, Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks, 2017. [Project page] [Dataset] [Code and models]

M. Elgharib, Z. Wu, R. Radke and V. Saligarama, Distributed Video Analytics and Anomaly Detection, 2014.

M. Elgharib, V. Golyanik, T. Leimkühler, M. Habermann, C. Theobalt, Advanced Topics in Neural Rendering and Reconstruction, University of Saarland, 2024.

C. Theobalt, V. Golyanik, M. Elgharib, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2023.

C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2021.

C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2020.

C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2019.

M. Elgharib, W. Matusik, S.-H. Bae and M. Hefeeda, View generation from a single image using fully convolutional neural networks, 2018.

M. Elgharib, M. Hefeeda, W. Freeman and F. Durand, Method for dynamic video magnification, 2015.

M. Hefeeda, K. Calagari, M. Elgharib, W. Matusik, P. Didyk and A. Kaspar, 2D-to-3D video frame conversion, 2015.

BBC News: What is deep learning ?

MIT News: Amplifying Small Motions in Large Motions.

MIT News: System Automatically converts 2-D video to 3-D. Featured on MIT main page!

2011: PhD in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Full paid non-EU scholarship from IRCSET.

2008: B.A.I in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Gold Medalist.

Reviewer for CVPR, ICCV, ECCV, SIGGAPH, SIGGRAPH-Asia, NeurIPS, PAMI, TIPS, 3DV, ACCV, and others.

Hobbies: I enjoy travelling, sightseeing, walking, hiking, natural scenery, watching documentaries and catching up with friends!